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(54) Signal converting apparatus and signal converting method 

(57) The signal converting apparatus and method 
predictively produces highly accurate Interpolated pix- 
els In accordance with a classification which precisely 
reflects a variety of signal characteristics of inputted 
video signals to provide a high resolution video signal. 
An activity is evaluated and classified (12) for each 
block of an inputted video signal (81). Stepwise classifi- 
cations (p0-p2), (Fig 5. 22-24) are executed on each 
block of the inputted video signal (SI) and a classifica- 
tion (01) is selected in accordance with an activity code 
(cO) obtained as a result of the activity classification 
(21). In this way, the accuracy of subsequent classifica- 
tions can be increased, reflecting the activity character- 
istic of each t>lock of the inputted video signal (S1), thus 
achieving, as a whole, a highly accurate classification of 
the inputted video signal (S1). Appropriate prediction 
coefficients (d1) stored in a ROM (14) are read out 
based on the activity code (cO) and the class code (c1) 
for each trfock of the inputted video signal (S1). The 
coefficients are used in calculators (13) to produce 
highly accurate interpolated pixel values (d2<15) which 
are selected by a selector 15 to provide a high resolu- 
tion video signal (S2). 
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Description 

This invention relates to a signal converting apparatus and to a signal converting method. Embodiments of the 
invention are applicable to upconvertors for converting standard definition signals (SD). for exanrple NTSC signals or 
5 the like, to high definition signals (HD), for example High Vision or the like. 

Heretofore, this type of upconvertors perform frequency interpolation on SD video signals to increase the number 
of pixels In the SD video signals to produce HD video signals. For example, as shown In Rg. 1 , such upconvertors per- 
form double frequency Interpolation respectively in the horizontal direction and in the vertical direction on an SD video 
signal composed of pixels represented by large "O" marks and large "a" marks on scanning lines 1 of an HD image to 
10 produce an HD video signal composed of pixels represented by small " O " marks and small "a" marks. 

As an exanrple of performing interpolation using an upconvertor, there is a method wNch produces HD pixels at 
four different positions from field data of an SD video signal. For example, taking an SD pixel represented by a mark 
" @ " in consideration, HD pixels at four different positions mode 1 . mode 2, mode 3 and mode 4 in the vicinity of the SD 
pixel "@" are produced by interpolation of environmental SD pixels. An intra-space two-dimensional non-separable fil- 
75 ter 2 shown In Fig. 2 and a horizontal/vertical separable fitter 3 shown in Fig. 3 are used for this operation as interpola- 
tion filters. 

The two-dimensional non-separable filter 2 employs two-dimensional filters 4 A to 4D to independentiy execute 
interpolation to generate HD pixels at four positions mode 1, mode 2, mode 3. nrtode 4, and converts the respective 
interpolated results into a serial form In a selector 5 to produce an HD video signal. 

20 Ihe horizontal/vertical separable filter 3 executes interpolation for pixels at positions mode 1 , mode 3 with a vertical 
interpolation filter 6A and executes Interpolation for pixels at positions nrxxJe 2, mode 4 with a vertical interpolation filter 
6B to produce data on two scanning lines of an HD video signal. Then, the filter 3 uses horizontal interpolation filters 
7A and 7B on the respective scanning lines to Interpolate HD pixels at the four positions, and converts the interpolated 
results Into a serial form in a selector 8 to produce an HD video signal. 

25 While the conventional upconvertor as described above employs an ideal filter as an interpolation filter, the spatial 
definition of a resulting HD video signal remains identical to that of an original SD video signal although the number of 
pixels are Increased in the HD video signal. Also, the conventional upconvertor has a problem that it can only produce 
an HD video signal having a lower definition than that of an original SD video signal since an ideal filter cannot be used 
in practice. 

30 As a method for solving the problem mentioned above, a classification adaptive processing method which classifies 
an inputted SD video signal into several classes on the basis of the characteristics thereof and uses prediction coeffi- 
cients conprising prediction data previously generated by learning for each class to produce an HD video signal with a 
high definition is proposed. For example, such method has been proposed by the applicant of this irrvention in the ^ec- 
ification and drawings of U.S. application Serial No. 08/061.730 filed in May 17. 1993. 

35 However, this classification adaptive processing method inplies a problem that a prediction accuracy for the HD 
video signal produced thereby is degraded unless an appropriate classification is carried out in accordance with the 
characteristics of an inputted SD video signal when prediction coefficients are generated by learning. In other words, 
without sufficient classification capability. HD video signals which would essentially classified into different classes may 
be grouped into the same class. Thus, prediction coefficients generated by learning will predict an average value of HD 

40 video signals of different nature, resulting in a degraded definition recovering capability. 

One aspect of this invention provides a signal converting apparatus for converting a first inputted video signal Into 
a second video signal different from the first video signal, conprising: means for evaluating an intra-space activity of the 
first video signal and outputting an activity code; means for executing stepwise classifications on the basis of the activity 
code and outputting a dass code on the basis of the result of the classification; a prediction coefficient memory for stdr- 

45 ing prediction coeffiderrts for predictively producing the second video signal by using the first video signal; and means 
for performing a prediction calculation on the first inputted video signal by using the prediction coefficient read from the 
prediction coefficient memory in accordance with the activity code and/or the class code to produce the second video 
signal. 

The first video signal may be a lower definition video signal, and the second video signal may be a higher definition 
50 video signal which is higher definition than the lower definition video signal. 

The second video signal may be a video signal that has the number of pixels more than the first video signal. 

The means for producing tfie activity code may evaluate the intra-space activity and activity In a temporal direction 
of the first video signal to output the activity code. 

The means for producing the class code may set a plurality of different pixel patterns to the first video signal, select 
55 a pixel pattern from a plurality of ttie set pixel patterns in accordance with the activity code, and classify the first video 
signal by using tiie selected pixel pattern to output the class code. 

In an embodiment of the invention, an irrti-a-space activity of an inputted video signal is evaluated, and stepwise 
classifications are executed for the Inputted video signal in accordance with the obtained activity code. In this way. sub- 
sequent classifications can increase the accuracy, reflecting the intra-space activity characteristic of the inputted video 
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signal. In addition, since the result of the previous classification is reflected to the stepwise sut^sequent classifications, 
the classification can be executed with a high accuracy. 

Prediction coefficients appropriate to the inputted video signal for each block are read based on at least a dass 
code thus obtained to produce highly accurate interpolated pixels, thus providing a video signal at a higher resolution. 

5 Another aspect of the present invention provides a signal converting apparatus for converting a first inputted video 

signal into a second video signal different from the first video signal, comprising: means for evaluating the intra-space 
activity for the first video signal to output an activity code; means for exiecuting stepwise classifications on the basts of 
the activity code to output a class code on the basis of the result of the classifications; and means, including a prediction 
value storing memory which stores the prediction value generated as an interpolation pixel signal of the first video sig- 

10 nal, for reading ard outputting a prediction value in accordance with the activity code arxl/or the dass code. 

A further aspect of the present invention provides a signal converting method for converting the inputted first video 
signal into the second video signal different from the first video signal wherein the intra-space activity of the first video 
signal is evaluated to output the activity code, stepwise classificatiorts is executed in accordance with the activity code, 
and a dass code is outputted in accordance with the result of the classifications. Then, prediction coefficients stored in 

IS the prediction coefficient memory for predictively producing the second video signal by using the first video signal in 
accordance with the activity code and/or the class code, the prediction calculation is performed on the first inputted 
video signal using the read prediction coefficients, and a prediction calculation value is outputted as the second video 
signal. 

A yet further aspect of the present invention provides a signal converting method for converting the inputted first 
20 video signal into the second video signal different from the first video signal, wherein the intra-space activity of the first 

video signal is evaluated to output an activity code, stepwise classifications are executed on the basis of the activity 

code, a dass code is outputted on the basis of the result of the dassif ications, a prediction value stored in tine prediction 

value memory in accordance with the activity code and/or the dass code is read, and the prediction value produced as 

the interpolation pixel signal of the f irst video signal is outputted. 
25 Embodiments of the invention seek to provide a signal converting apparatus and signal converting metiiod which 

are capal^le of converting lower definition video signals into higher definition video signals by appropriate dassif ications 

corresponding to a variety of signal characteristics of irq^utted video signals. 

A better urKierstandtng of the invention will become more apparent from the following detailed description when 

read in conjunction with the accompanying drawings in which: 

30 

Fig. 1 is a schematic diagram explaining the relation between SD video signal and HD video signal; 

Rg. 2 is a block diagram showing a conventional two<lirnensional non-separable interpolation filter; 

Rg. 3 is a block diagram showing a conventional vertical/horizontal separatee interpolation filter; 

Rg. 4 is a block diagram sfiowing an upconvertor comprising a two-dimensional non-separable filter according to 
35 the present invention; 

Rg. 5 is a fc>lock diagram showing the configuration of a classification unit shown in Rg. 4; 

Rg. 6 is a block diagram showing the configuration of an activity determinatic^ unit shown in Rg. 5; 

Rg. 7 is a schematic diagram showing an exernplary positioning of SD pixels; 

Rgs. 8A, 8B and 8C are schematic diagrams showing dass tap patterns for classification unit; 
40 Rg. 9 is a schematic diagram showing the prediction taps of learning data; 

Fig. 10 is a flow chart showing a prediction coefficient learning procedure; 

Rg. 11 is a schematic diagram explaining a hierarchical structure in a prediction coeffident ROM; 

Rg. 12 is a block diagram explaining an activity dassif ication unit in an upconvertor according to a second emt>od- 

imerrt; 

45 Fig. 13 is a graph showing a frequency distribution based on the level of ADRC code value; 
Fig. 1 4 is a block diagram explaining an upconvertor according to a sixth embodiment; 
Rgs. 15 A. 156, 15C. 15D and 15E are schematic diagrams explaining one<limensional Laplacian fflters; 
Rg. 16 is a tat>le explaining indexes for a dass code of first-step classification unit; 

Rgs. 17A, 17B, 17C, and 17D are schematic diagrams explaining prediction tap patterns having a characteristic in 
50 a horizontal direction as a classification result of dassrfication unit; 

Figs. ISA. 18B, 18C, and 18D are schematic diagrams explaining prediction tap patterns having a characteristic in 
a vertical direction as a das^ification result of classification unit; 

Rgs. 19A, 19B, 19C. 19D, 20A. 20B, 20C, and 20D are schematic diagrams explaining prediction tap patterns hav- 
ing a characteristic in an oblique direction as a classification result of classification unit; 
55 Rg. 21 is a block diagram explaining an upconvertor according to a seventh embodiment; 

Rg. 22 is a block diagram explaining a dassif tcation unit according to an aghth enr^odiment; 
Rgs. 23A. 23B, 23C. and 23D are schematic diagrams explaining class tap patterns in classification units; 
Figs. 24A, 24B, and 24C are schematic diagrams explaining dass tap patterns in classification units; 
Fig. 25 is a block diagram explaining an LqDConvertor according to a ninth embodiment; 
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Rg. 26 is a schematic diagram explaining a class generating trfock data used for generating a class code in a clas- 
sification unit according to an ninth embodiment; 

Rg. 27 is a schematic diagram explaining a hierarchical structure of class code according to an ninth embodiment; 
Fig. 28 is a schematic diagram explaining prediction tap patterns in a prediction calculation unit according to an 
5 ninth errtbodiment; 

Rg. 29 is a block diagram showing an upconvertor based on a verticalZhorizonta! separable filter according to the 
present invention: and 

Rg. 30 is a block diagram showing an upconvertor for generating HD pixel signals based on prediction values 
according to the present invention. 

10 

(1) Rrst Embodiment 

Fig. 4 shows as a whole which employs a two-dimensional non-separable filter utilizing classification adaptive 
processing to produce an HD video signal from an SD video signal. An SD video signal SI inputted to the upconvertor 

15 10 through an input terminal IN is supplied to a classification unit 12 in a block-by-block scheme conposed of predeter- 
mined number of pixels in which a remarked SD pixel is the center of the pixels, then the SD video signal SI is supplied 
to a prediction calculation unit 13. The classification unit 12 generates a class code dO of the remarked SD pixel on the 
basis of the diaracteristics of the SD pixels of the SD video signal Si in the vicinity of the remarked pixe! of the inputted 
SD video signal SI . The class code dO is output as address data to a prediction coefficient ROM (Read Only r^emory) 

20 1 4 which serves as storage means. 

The prediction coefficient ROM 14 stores prediction coefficients, which has been previously obtained by learning 
and which is used to predictively calculate HD interpolated pixels for producing a high definition video signal from a low 
definition resolution video signal, as prediction data d1 corresponding to the class code dO. The prediction coefficient 
ROM 14 reads the prediction data d1 using the class code dO as address data, and supplies it to the prediction calcu- 

25 lation unit 13. The prediction calculation unit 1 3 executes a predetermined prediction calculation on the SD video signal 
S^ using the prediction data d1 to produce HD interpolated pixels from the SD video sjgnal S^. The SD video signal S^ 
is supplied via a delay unit which is not shown to the prediction calculation unit 1 3. A delay time of the delay unit corre- 
sponds to the time to finish supplying the prediction data d1 to the prediction calculation unit 13. 

The prediction calculation unit 13 is connposed of four prediction calculators 13A to 13D. The respective prediction 

30 calculators 13A to 13D execute a product sum calculation using the prediction data d1 on the SD video signal S^. 
Thereby, the prediction calculators 13A to 13D produce prediction values d2. d3, d4. d5 for HD interpolated pixels cor- 
responding to pixels at four different positions mode 1 . mode 2, mode 3. rrxxJe 4 on a scanning line 1 . respectively. The 
respective HD interpolated pixels d2. d3, d4. d5 produced in the corresponding prediction calculators 13 A to 13D are 
supplied to a selector 15. The selector 15 rearranges the respective prediction values d2, d3. d4, d5 into time-series 

35 data, using a buffer memory (not shown), which is then outputted from an output terminal OUT as an HD video signal 
S2. 

Fig. 5 shows the configuration of the classification unit 12 shown in Rg. 4. As shown in Fig. 5, the SD video signal 
Si inputted through an input terminal IN is supplied to an activity classification unit 21 . Then, in the activity classification 
unit 21 . for exanple. the spatial activity is classified for each block composed of 9 pixels of 3 x 3, with a remarked pixel 

40 being centered, to evaluate and determine the characteristics of each block. The activity classification unit 21 produces 
a class code cO on the basis of the classif ication and the evaluation of the spatial activity to output the dass code cO to 
a selector 25 and an ADRC (adaptive dynamic range coding) classification unit 26. 

In addition, the SD video signal Si is parallelly supplied to a wide region tap selector 22. a standard tap selector 23 
and a nanrow region tap selector 24 for setting three different types of pixel tap patterns. The wide region tap selector 

45 22. the standard tap selector 23 and the nan-ow region tap selector 24 each select tap patterns pO. pi , and p2 corre- 
sponding to the space dasses for the inputted SD video signal S-i . 

Rg. 6 shows the cortfiguration of the activity classification unit 21 shown in Rg. 5. As shown in Rg. 6, the SD video 
signal S^ inputted from the input terminal IN is outputted to a processing unit 30 at first Then, the processing unit 30 
detects a dynamic range OR In a plurality of the SD pixels in which the remarked SD pixel is the center of the inputted 

50 SD video signal. TTie dynamic range DR is defined for example, using a maximum value MAX and a minimum value 
M IN in a neighlx)ring region consisting of nine pixels in which the remarked pixel is the center (shown by "®T as 
shown in Rg. 7 by the following expression (1 ): 

DR = MAX-M1N (1) 

55 

The dynamic range DR of the plurality of the SD pixels in which the remarked SD pixel is the center is output to a thresh- 
oW determination unit 31. The dynamic range DR is compared with a predetermined threshold In the threshold deter- 
mination unit 31. As a result the threshold determination unit 31 outputs a class code cO produced by comparing the 
dynamic range with the threshold. In short, the spatial activity is determined by determining the size of the dynamic 
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range by three size (that is. the spatial activity is decided to one of three step of high, middle and low) by the threshold 
value processing in the threshold determination unit 31 . Then the determined result is output as a class code cO repre- 
sented by two bits. It is generally thought that the spatial activity is high as the dynamic range DR is large, and the spa- 
tial activity is low as the dynamic range DR is small. In this way. the activity classification unit 21 executes af irst step of 

5 a classification teased on the dynamic range. 

Next a next step of a classification in the wide region tap selector 22, the standard tap selector 23. the narrow 
region tap selector 24 and the ADRC classification unit 26 will be describkj specrficaliy. 

First, among the foregoing three types of class tap pattern selectors, the standard tap selector 23 takes in consid- 
eration the standard intra-space variations of the inputted SD video signal and selects an ordinary class tap pattern 

10 as shown in Rg. 8B, On the contrary, the wide region tap selector 22 takes in consideration the regular irrtra-space var- 
iations of the inputted SD video signal Si . That is, the wide region tap selector 22 selects a class tap pattern for a wide 
region as shown in Fig. 8A. Further, the narrow region tap selector 24 takes in consideration the irregular intra-space 
variations of the inputted SD video signal S^ and selects a dass tap pattem for a narrow region as shown in Fig. 8C for 
the irregular signal variations. 

15 The wide region tap selector 22. the standard tap selector 23 and the narrow region tap selector 24 respectively 
supply class tap patterns pO, p1 and p2 respectively selected thereby to the selector 25. The selector 25 selects one of 
the class tap patterns pO. pi and p2 in response to a class code cO sent thereto from the activity classification unit 21 
as a selection control signal, and supplies the selected class tap pattern as a class tap pattern p3 to the ADRC classi- 
fication unit 26. That is. the selector 25 selects the class tap pattern pO from the wide region tap selector 22 when the 

20 class code cO indicates that the spatial activity is low. to the comrary. the selector 25 selects the dass tap pattern p2 
from the narrow region tap selector 24 when the class code cO indicates that the spatial activity is high. 

The ADRC dassification unit 26 sets the number of requantization bits "k" in response to the class code cO which 
is used as a control signal. The dass code cO has been generated in accordance with the dynamic range DR of the 
inputted SD video signal Si . In this way, the level resolution capability can be set differentiy for each tap in the tap pat- 

25 tern p3 selected for a space class, depending upon the dynamic range DR of the SD video signal S^. 

The ADRC re-quantizes pixels with a quantization step size defined as the re-quantization. An ADRC code c1 (**d'* 
is used in the following equation in accordance with the number of SD pixels *1" in the dass tap pattern) is represented, 
using the dynamic range DR. the number of re-quantization bits an SD pixel Xj. and a minimum pixel level MIN in its 
neighboring region, by the following expression: 
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Xj-MIN 



The change of the level resdution capability for a tap pattern of a space class is carried out by changing the number 
of re-quantization bits "k" in tiie ADRC calculation represented by expression (2) in accordance with tiie dass code cO. 
In this way. the level resolution capability can be adaptively changed and set in accordance with the dynamic range DR 
of an inputted signal. That is. the bigger the dynamic range DR becomes, the more the ADRC dassification unit 26 set 
40 the level resolution capability in detail. 

The dassification unit 12 thus generates a class code dO composed of the dass code cO and an ADRC code c1. 
The class code dO is supplied to the prediction coeffident ROM 14 at a subsequent stage as address data. 

The prediction coeffident ROM 14 reads the class code dO composed of a combination of the class code cO and 
the ADRC code c1 as address data, and supplies prediction data d1 to be used to produce HD interpolated pixels to 
45 the prediction calculation unit 13. The respective prediction calculators 13A to 13D execute a predidion calculation 
using SD pixels Xj comprising the SD video signal S^ and prediction coefficients Wj comprising prediction data d1 for 
each class to produce predicted pixels y' for HD interpolated pixels corresponding to the positions mode 1 to mode 4 
on the scanning line 1 . 

The SD pixels Xj used in this event are formed, for example, of thirteen prediction tap data comprising a remarked 
50 pixel (indicated by "®T arwj surrourKling pixels (indicated by "O") positioned as shown in Rg. 9. Therefore, in this 
case, the prediction coefficients Wj conprise thirteen prediction coeffidents Ibr the respective priediction calculation 
units. Moreover, the SD pixels used in the respective prediction calculation units 13A to 13D are identical with one 
another, but the prediction coefficients Wj from the prediction coefficient ROM 1 4 are different in the respective predic- 
tion calculation units 13A to 13D, so that the prediction coefficient ROM 14 stores four groups of the prediction coeffi- 
55 dents comprising thirteen prediction coefficients corresponding to one dass. 

The predicted pixels / for the HD interpolated pixels are transformed and produced, using the foregoing thirteen 
SD pixels Xj and the predidion coefficients W|. by the following expression (3): 
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y^3 



Wj X Xj = X X^ + " 



+ W^3 X X^3 



(3) 



5 The respective prediction calculation units 13A to 13D executes the prediction calculation by the expression (3) using 
the SD pixels and the prediction coefficients respectively supplied, and produces the HD interpolation pixels. 

The prediction coefficients Wj used herein have been previously generated by learning and stored in the prediction 
coefficient ROM 13. 

Next, a learning procedure for generating prediction coefficients for each dass stored in the prediction coefficient 
10 ROM 14 will be described by referring a flow chart shown in Fig. 10. 

The prediction coefficients are generated in accordance with a prediction coefficient learning procedure shown in 
Fig. 10. Upon starting the prediction coefficient learning procedure at step SP1. learning data corresponding to previ- 
ously known images are first generated at step SP2 for learning prediction coefficients Wj. 

Specifically, in the HD image shown in Rg. 1 . an HD interpolated pixel is designated as an HD remarked pixel, and 
15 this HD remarked pixel is expressed by a linear primary combination model using prediction coefficients by a set of 
learning data comprising surrounding HD interpolated pixels and SD pixels. The prediction coefficients used in this 
event are calculated using a least squares method for each class. In addition, in generating learning data as descrit>ed 
above, if a plurality of images are used, instead of a single image, to generate a multiplicity of learning data, more accu- 
rate prediction coefficients can be generated. 
20 It Is determined at step SP3 whether or not a sufficient number of learning data have been generated at step SP2 
for ot>taining the prediction coefficients. If it is determined thiat the number of generated learning data is less than a 
required number, the prediction coeff icierrt learning procedure proceeds to st^ SP4. 

At step SP4. class learning data are classified. The classification is performed in such a manner that a local flat- 
ness is first detected for learning sampling data, and pixels used for the classification are selected in accordance with 
25 the detection results. In this way. pixels exhibiting small changes of the input signal are removed from data to be 
learned, so that the influence of noise can be eliminated. The classification of the class learning data is carried out by 
executing the same processing as that used for classifying the inputted SD video signal S^ . 

More specifically, the classification of the class learning data begins with the classification and evaluation of the 
dynamic range DR of learning data to set a class code cO. Subsequently, a tap pattern p3 is selected from three kinds 
30 of wkje region, standard and narrow region tap patterns based on the class code cO as a space dass. Then, as shown 
in Rg. 1 1 . the class code cO thus generated is combined with an ADRC code c1 to set a class code dO. and this class 
code dO is stored in ROM in correspondence to prediction data d1. 

Subsequently, the prediction coefficient learning procedure forms at step SP5 a normalization equation for each 
class teased on the classified learning data. 
35 The processing at step SP5 will be specifically explained. However, for generalization, described below is a case 
where "n" sampling pixels exist as learning data. First, the relationship between pixel levels x^ x^ of respective sam- 
pling pixels and a pixel level *'y" previous to a subsample of a remarked interpolated pixel is expressed for each dass 
by a prediction expression represented by a linear primary combination model using **n" taps of prediction coefficients 
w^ Wp. The prediction expression is given by the following expression (4): 

40 

i=n 

y = ^ w, X Xj (4) 
i=1 

45 The prediction coefficients w^ w^ in the expression (4) are calculated to predict the pixel level "y". 

Next, an example will be given for showing how to generate the prediction coefficients w^. ..... w^ by a least squares 

method. The least squares method is applied as follows. 

As a generalized example, the following observation expression (5) is considered where X represents a set of input 
data, "w" a set of prediction coefficients, and "Y" a set of predicted values. 

"50 

XW = Y 

where, 

55 
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X = 



Xll Xi2 
X2X X22 



X2n 



Xml Xm2 • - * Xmn 



W = 



W2 
Wn 



, Y = 



yi 

y2 



(5) 
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The least squares method is applied to data cx)Ilected by the observation expression given by expression (5). In the 
example given by expression (5), "n" is equal to "13". and "m" represents the number of leaming data. 

Rrst based on the observation expression of the expression (5). the following residual expression (6) is consid- 
15 ered: 



XW=Y+E 



where. 



20 



25 



(6) 



It can be thought from the residual expression given by the expression (6) that the most probable value of each Wj 
is derived when a condition for minimizing the solution of the following equation (7) is satisfied. 
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i=1 



(7) 



More specifically, when a partial differential expression by Wj of expression (7) is expressed by the following expres- 
sion (8): 



de. 



de. 



e 1 + e2 + 
^ dw, oWj 



de 



m 

dWi 



em -^r-^ = 0 (i= 1.2 n) 



(8) 



40 conditions to the number of "n" are considered based on "i" in the expression (8), and w^, W2. Wn satisfying these 

conditions may be calculated. Thus, the following expression (9) is derived from the residual expression (6). 



3ei 



dej 



45 



dw, " dWg ~ dw, 



= X,n, (1 = 1.2 n) 



(9) 



50 



From the expression (9) and the expression (8). the following expression (10) is derived: 

n n n 
£e,x„=0.£e,Xi2 = 0. J^e^x^^^O 

i«1 1=1 i=1 



(10) 



Then, from the expression (6) and the expression (10). the following normalization expression (11) is derived: 
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( Z Xji Xji)wi + ( ^ Xjl Xj2)W2 + 



{ Z Xjl Xjn)Wn = ( Z ^jl Yj) 



j = l 



( Z Xji)wi + ( Z Xj2 Xj2)W2 + 



3=1 



+ ( Z Xj2 Xjn)Wn = ( Z Xj2 Yj) 



3 = 1 



j = l 



( Z Xjn Xji)wi + ( Z ^jn Xj2)w 



2 + 



3 = 1 



3 = 1 



+ ( Z Xjn Xjn)Wn = ( Z Xjn Yj) 
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Since the number of normalization equations given by the expression (1 1) equal to the number "n" of unknowns 
can be formed, the most probable value of each Wj can be calculated from these normalization expression. 

The normalization expressions can be solved using a sweeping-out method (Gauss- Jordan's elimination method). 
30 The prediction coefficient learning procedure repeats a loop of steps SP2-SP3-SP4-SP5-SP2 until the same 
number of the normalization expressions as the number "n" of unknowns are formed for calculating indefinite coeffi- 
cients w^, for each class. 

When the required number of normalization expressions are thus formed, an affirmative result is derived for a 
determination at step SP3 as to whether or not learning data have been ended, followed by the procedure proceeding 
35 to a determination of prediction coefficients at step SP6. 

At step SP6, the normalization expressions given by the expression (11) are solved to determine the prediction 

coefficients w^ Wn for each class. The prediction coefficients thus determined are stored at the next step SP7 in a 

storage means such as ROM which has its storage area divided for each class. At this case, four group of the prediction 
coefficients comprising the prediction coefficients w^. ...,Wn respectively corresponding to the prediction calculators 13A 
40 to 1 3D are stored with respect to one class code. By the foregoing learning procedure, the prediction coefficients for the 
classification predictive processing are generated, followed by the termination of the prediction coefficient learning pro- 
cedure at the next step SP8. 

Next, the operation of the aforementioned upconvertor 10 of the first embodiment and each unit of the upconvertor 
will be descrftjed. An SD video signal inputted to the upconvertor 10 through the input terminal IN is first supplied 

45 parallelly to the classification unit 12 and the prediction coefficient calculation unit 13. The classification unit 12 gener- 
ates a class code dO fc»ased on the SD video signal and supplies the generated class code dO to the prediction coef- 
ficient ROM 14. The prediction coefficient ROM 14 reads prediction data d1 previously obtained by learning in 
accordance with the class code dO, and supplied it to the prediction coefficient calculation unit 13. The prediction coef- 
ficient calculation unit 13 produces HD interpolated pixels corresponding to four positions (mode 1 to mode 4) on a 

50 scanning line 1 based on the SD video signal Si sent from the input terminal IN and the prediction data d1 supplied 
from the prediction coefficient ROM 14 in the respective prediction calculators 13A to 1 3D. 

In the classification unit 12. the activity classification unit 21 first detects a dynamic range DR of a plurality of SD 
pixels in which the remarked SD pixel of the inputted SD video signal Si is the center, and conrrpares the dynamic range 
DR with a predetermined threshold to output a class code cO. Generally, the spatial activity is high as the dynamic range 

55 is large, and conversely, the spatial activity is low as the dynamic range is small. 

Meanwhile, the inputted SD video signal S^ in block units is parallelly supplied to the wide region tap selector 22. 
the standard tap selector 23 and tine narrow region tap selector 24 for setting tinree different pixel tap patterns. And the 
wide region tap selector 22. tiie standard tap selector 23 and the narrow region tap selector 24 set tap patterns pO, pi 
and p2 for the respective space classes. 
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The selector 25, based on the class code cO. selects a dass tap pattern pO having a signal change over a relatively 
wide range as shown in Fig. 8A for an SD video signal having a small dynamic range DR and a low activity in order 
to reflect a slow signal change to the dass. On the other hand, the selector 25 selects a class tap pattern p2 having a 
signal change over a narrow region as shown In Fig. 8C for an SD video signal Si having a large dynamic range DR 

5 and a high activity in order to express a signal change in a narrow region with a largest possible number of classes. In 
this way, depending upon the signal characteristics in view of the dynamic range DR, the selector 25 selects and sup- 
plies a space class represented by a tap pattern p3 reflecting a signal diange of an associated SD video signal Si to 
the ADRC dassification unit 26 at the next stage. 

The ADRC classification unit 26. using the dass code cO as a control signal, sets a small value to the number of 

10 requantization bits "k" of each tap for the space classification for an SD video signal Si having a small dynamic range 
DR. This results in redudng the level resolution capability of each tap, thus outputting an ADRC code c1 on the assump- 
tion that the SD video signal Si is stable. On the other hand, the ADRC dassification unit 26 sets a larger value to the 
nunrt^er of requantization bits ^^k*" of each tap for the space dassification for an SD video signal S-i having a large 
dynamic range DR. In order to output an ADRC code c1 with a higher level resolution capability. In this way, an unstable 

75 signal change of the SD video signal S^ having a large dynamic range DR and a high activity can be reflected to the 
class. 

As described atx>ve, the classification unit 12 changes a tap pattern of pixels used for the classification in accord- 
ance with the dynamic range DR of an inputted SD video signal S^. and also changes the number of re-quantization 
bits ^'k" of each tap for the classification to adaptively set the level resolution capability. This can provide an appropriate 
20 Classification in accordance with the characteristics of the dynamic range of the inputted SD video signal S^. 

The dassification unit 12 combines the dass code cO with the ADRC code c1 to generate a dass code dO which is 
supplied to the prediction coeffident ROM 1 4 at the next stage. In the prediction coefficient ROM 1 4. prediction data d1 
is read based on the dass code dO and supplied to the prediction calculation unit 13. The prediction calculation unit 13 
produces HD interpolation pixels by transforming SD pixels into HD interpolation pixels using the prediction data d1. 
25 The HD Interpolated pixels are supplied to the selector and rearranged time-series at the selector 15 and output as HD 
video signal. Thus, the selected predidion data d1 reflects the characteristics pf the inputted SD video signal S^ in 
terms of the dynamic range DR, thereby making It possible to improve the accuracy of HD interpolated pixels, produced 
by transforming SD pixels, and Inprove the spatial resolution capability of an HD video signal S2. 

According to the foregoing embodiment, an SD video signal S-i inputted to the upconvertor 10 undergoes a deter- 
30 mination in the activity dassification unit 21 as to whether its dynamic range DR Is larger or smaller than a threshold 
value. Then, a tap pattern suitable for the characteristics of the SD video signal S-i In terms of the dynamic range DR 
can be set from tap patterns for three kinds of space dasses (a wide region tap pattern, a standard region tap pattern, 
or a narrow region tap pattern), based on a class code dO generated as the result of the deterrhination. It Is therefore 
possible to set a dass tap pattern which reflects the characteristics of the inputted SD video signal Si in terms of the 
35 dynamic range DR. 

Also, according to the foregoing emtx)diment. the numt^r of re-quantization bits "k" of each tap for the space clas- 
sification is changed in accordance witii a dass code c1 to change the level resolution capability of each tap. thereby 
making it possit)le to reflect stable or instable signal changes to the classification by use of the level resolution capability 
of the tap. In this way. the inputted SD video signal S^ Is appropriately classified witii a tap pattern and a level resdution 
40 capability of the tap set in accordance with the dynamic range DR of the inputted SD video signal S^ , making it possikMe 
to produce an HD video signal S2 having a high spatial resolution which reflects the signal characteristics of the Inputted 
SD video signal Si . 

(2) Second Embodiment 

45 

Rg. 12 shows an activity dassification unit 35 of the upconvertor according to a second embodiment. The activity 
classification unit 35 evaluates an Intra-space activity of an SD video signal S^ , comprising a plurality of SD pixels in 
which a remarked pixel is the center, inputted thereto in block units and classifies the remarked pixel in accordance with 
its characteristics. 

so The SD signal Si inputted from an input terminal IN is supplied to the ADRC classification unit 36 which executes 
a classification based on ADRC for the SD video signal S^ comprising a plurality of the SD pixels in which the remarked 
pixel Is the center. 

An ADRC code "c" (It should be noted that q is used for the AD RC code c in the expression (1 2) in order to conform 
to the nunrd^er i of SD pixels in a class tap pattern) outputted from the ADRC classification unit 36 is generated from a 
55 dynamic range DR, a number of re-quantization bits "k", an SD pixel Xj, and a minimum pixel level MIN within a neigh- 
. boring region of tiie SD pixel as expressed by the following expression (12): 
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c, = ^^ (12) 
2" 

5 similarly to the first embodiment 

The ADRC code "c" generated in the ADRC classification unit 36 is supplied to a post-processing unit 37 at the next 
stage. The post-processing unit 37 represents a variation degree of a level distribution pattern indicated by the ADRC 
code "c". for example, a standard deviation a for the ADRC code "c" is calculated. And the post-processing unit 37 sup- 
plies the calculated standard deviation a for the ADRC code "c" to a threshold determination unit 38 at the next stage. 

10 The thr^hold determination unit 38 generates and output a dass code cO by conparing the standard deviation a for 
the ADRC code "c" with a threshold for determination. The standard deviation a for the ADRC code "c" is expressed by 
the following expression (13) using the ADRC code "ci". an average value "ca" of the ADRC code "d", and a number of 
the ADRC code "n". 



IS 



'V t=o 



f (13) 



The selector 25 shown in Rg. 5 uses the dass code cO thus generated to select a tap pattern p3 for a space class, 
20 similarly to the first enrtoodiment Also, the ADRC dassification unit 26 adaptively sets a level resolution of each tap 
t>ased on the class code cO. In this way. the tap pattern and level resolution are set for the space dass based on the 
result of classifying the spatial activity of the inputted SD video signal Si. thus produdng similar effects to the first 
eniKxjiment. 



25 (3) Third Embodiment 

As third embodiment, for classification, the standard deviation a may be calculated, for example, in view of a data 
distribution of nine pixels around a remarked pixel of an inputted SD video signal S^ shown in Rg. 7. and generated an 
class code cO by a threshold determination for the calculated standard deviation cr. 

30 In other words, in the processor 35 of the activity dassification unit 21 shown in Rg. 6, the standard deviation a is 
calculated in view of the data distribution of nine pixels around the inputted remarked pixel. 

In short, in the processor 35, the SD signal S^ inputted from the input terminal IN is supplied, the standard deviation 
a is calculated in view of the data distribution of nine pixels around the inputted remarked pixel And the processor 30 
supplies the calculated standard deviation a to the threshold determination unit 31 at next stage. The threshold deter- 

35 mination 38 generates and outputs a dass code cO by a threshold determination of the standard deviation cr. The stand- 
ard deviation a is expressed by the following expression (14) using the SD pixels xi, an average value xa in a 
neighljoring region , and a number of pixels **n'' in the neighboring region. 



a= ^i:(x,-xj^ (14) 

In this way. in the third emtjodiment, the dassification is executed by a threshold determination using this standard devi- 
ation a. In general, the spatial activity is high as the standard deviation is large, and conversely, the spatial activity is 
45 low as the standard deviation is small. Therefore, by changing a space dass tap pattern and the level resolution capa- 
bility of a tap for the space classification based on the threshold determination of the standard deviation a, the same 
effects as the foregoing emt)odiments can be otstained. 

(4) Fourth Embodiment 

50 

Further, as the forth enrtbodiment. a frequency distribution table which registers the values of the ADRC code "c" 
represented a variation degree of a level distribution pattern indicated by the ADRC code "c" may be produced, and the 
a dass code cO data may be produced by a threshold determination using the generated frequency distrilxition table. 

More specifically, in a frequency distribution table as shown in Rg. 13, the number of ADRC codes "c" existing 
55 between threshold "0" arid threshold "1 " is counted, and the ratio of pixels existing in this region to pixels existing out of 
this region is determined for dassification. In this case, if the ADRC codes "c** are concentrated, for example, in a par- . 
ticular region, it may be determined that the spatial activity of the SD video signal is low. On the other hand, if the ADRC 
codes c are widely spread, it may be determined that the spatial activity is high. 

In other words, in an ADRC classification unit 36, the SD signal S^ inputted from the input terminal IN is supplied. 
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the plurality of the SD pixels in which the remarked pixel is the center are executed the classification based on the 
ADRC. 

The ADRC code "c" generated in the ADRC classification unit 36 is output to the post-processing unit 37 at the next 
stage. The post-processing unit 37 generates a frequency distribution table registered an ADRC code "c" as shown in 

5 Fig. 13 representing a variation degree of level distribution pattern based on the ADRC code "c". Then the post- 
Processing unit 37 outputs a data representing a frequency distribution table for the generated ADRC code "c" to a 
threshold determination unit 38 at the next stage, the threshold determination 38 counts the number of ADRC code 
existing between threshold "0" arKi threshold "1" in a frequency distribution table» thereby executes a threshold deter- 
mination, and generates and outputs a class code cO. 

10 Therefore, a space dass tap pattern and a level resolution capability of a tap for the space classification are 
changed based on the threshold value determination using the frequency distribution tat^e for this ADRC code "c". 
thereby obtaining the same effects as the foregoing embodiments. 

(5) Fifth Enr±KxJiment 

15 

Further, as the fifth embodiment, absolute values of differences of respective adjacent SD pixel values may be reg- 
istered in the frequency distribution table so as to produce a dass code **cO*' by evaluating a spatial activity using the 
frequency distribution table. Jn this case, the spatial activity is high as a large nunnber of pixels have large absolute dif- 
ference values, and conversely, the spatial activity is low as a large number of pixels have small absolute difference val- 
20 ues. 

An adjacent pixels difference calculation unit for calculating absolute values of differences of respective adjacent 
pixel values is set instead of the ADRC dassification unit 36 in the activity classification unit 35 shown in Rg. 12. 

in short, in the adjacent pixels difference calculation unit, an SD signal Si inputted from the input terminal IN is sup- 
plied, and difference between adjacent pixels Is calculated on plurality of adjacent SD pixels in which the remarked pixel 

25 is the center to generate absolute value of the calculated difference. The absolute difference value generated in the 
adjacent pixels difference calculation unit is su|:^lied to the post-processing unit 37 at next stage. The post-processing 
unit 37 produces a frequency distribution table, representing a variation degree of level distribution pattern based on the 
ADRC code "c", in which absolute difference value has been registered. Then, in the post-processing unit 37, data rep- 
resenting the frequency distribution table for the generated absolute difference value is output to a threshold determi- 

30 nation unit 38 at next stage. The threshdd determination unit 38 counts the number of pixels existing between threshold 
"0" and threshold in the frequency distribution table for absolute difference value, thereby a dass code cO is pro- 
duced arxl output by executing a threshold determination. 

Accordingly, a space dass tap pattern and a level resolution of a tap for the space dassification are changed on 
the basis of a thr^hold determination of the absolute difference values of the adjacent pixels, thereby producing the 

35 same effects as the foregoing embodiments. 

(6) Sixtii Enrd30diment 

Fig. 1 4 shows an upconvertor 50 according to a sixth embodiment The upconvertor 50 executes a first dimensional 

40 laplacian operations in plural directions in laplacian filters 51 A to 51 E of a first classification unit 50A. and a first-step 
classification by synthetically deciding tiie values. A class tap pattern of second classification unit 50B is set according 
to the dassification result of the first dassification unit 50A. Then the second dassification unit 50B executes a classi- 
fication using the class tap pattern. 

For the sixth embodiment, an activity of temporal direction and spatial direction are evaluated in the first step clas- 

45 sification. The structure of tiie sixtti embodiment will be described by using Rg. 14. 

An SD video signal S^ supplied from the input terminal IN is supplied to the classification unit 50A. Then the SD 
video signal S^ supplied to dassification unit 50A is supplied to the lapladan filters 51 A to 51 E and a delay drcuit 56B. 
The five different laplacian filters 51 A to 51 E perform lapladan operations in different directions from each filter, on each 
frame or each field of the inputted SD video signal Si. in order to output lapladan values LO to L4. 

so More specifically, tiie Lapladan operations are performed by the one-dimensional Laplacian filters 51 A to 51 E in 
the horizontal dir^on (Fig. 15A). the vertical direction (Fig. 158). a rigfitwardly dedining oblique direction (the direc- 
tion of a diagonal extending from the upper left end to the lower right end on the plane of the drawing) (Rg. 1 5C), a left- 
wardly declining oblique direction (the direction of a diagonal extending from the upper right end to the lower left end on 
the plane of the drawing) (Rg. 15D), and a temporal direction (Fig. 15E). as shown in Figs. 15A to 15E. 

55 Laplacian values U) to L3 resulting from the Lapladan operations performed by tiie Lapladan filters 51 A to 51 D are 
supplied to absdute value circuits 52A to 52D at the next stage, respectively The absolute value drcuits 52A to 52D 
calculate absolute values of tiie respective Lapladan values LO to L3 supplied tfiereto. And resulting al^solute values 
aO to a3 are supplied to a maximum value detector 53. The maximum value detector 53 detects a maxinruim value from 
the absolute values aO to a3. and compares the maximum valuie with a threshold value THO. With this operation, a flat- 
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ness of the inputted SD video signal Si is detected, a value (flatness values) L10 indicating the flatness represented in 
one bit is output. Further, the maximum value detector 53 outputs a value (maximum value detecting direction value) 
alO indicating a direction which a maximum value represented in two bits is detected. Simultaneously, the laplacian fil- 
ter 51 E outputs a laplacian value L4 in the tenrporal direction to an absolute value circuit 52E. The absolute value circuit 

5 52E calculates absolute value of the laplacian value L4. and supplies resulting at>solute value a4 to a comparator 54. 
The comparator 54 compares the absolute value a4 with a threshold value TH1. thereby outputs a value (temporal 
direction changing value) all indicating a change in the temporal direction represertted in one bit. Four bit data which 
is a combination of the temporal direction changing value a11, the above-mentioned maximum value a10 and the flat- 
ness value L10 is supplied to the classification unit SOB at the next stage as a control signal CT 

10 Since the laplacian value is basically a total of spatial differences between a remarked pixel and respective pixels 
on both sides, the laplacian value Is larger as adjacent pixels present large changes. The first classification unit 50A 
performs the one-dimensional laplacian filtering on the activity in a predetermined direction in the space in order to 
detect a direction in which an edge exists in a space, and roughly classifies its characteristic. 

The control signal CT outputted from the classification unit 50A is supplied to selectors 55A to 551 and a delay cir- 

15 cuit 56A. The selectors 55A to 551 are connected to a register array 57 through lines to which an SD data selected by 
the control signal CT is supplied. The register array 57 is supplied with SD video signal S^ of delayed several lines por- 
tions through the delay circuit 56B. The selectors 55A to 551 are selectively switched in response to the control signal 
CT, and selects SD image data supplied from the register array 57 in accordance with corresponding indices, and sup- 
plies pixel data for nine pixels to a one-bit ADRC classification unit 58 at the next stage. In short, the class tap pattern 

20 Of the classification unit 508 at the next stage is selected in accordance with the control signal CT. 

The ADRC dassrfication unit 5S uses a class tap pattern formed of nine taps selected by the selectors 55A to 551 
to execute one-bit ADRC classification to output an ADRC code "c" represented in nine bits. As a result, the classifica- 
tion unit 508 provides 512 (2^) types of different classes. 

Consequently, as 16 classes provided by the first classification unit 50 A is multiplied by 512 classes provided by 

25 the second classification unit 508, the upconvertor 50 can classify a unit block of SD video signal into 81 92 classes. By 
thus selecting an appropriate tap pattern by the ADRC classification at the next step in accordance with a class selected 
at the first step based on the spatial activity, a highly accurate classification can be accomplished at the next and sub- 
sequent steps, reflecting the spatial activity of the SD video signal. 

A prediction value calculation unit 50C at the stage subsequent to the classification unit 50B is composed of a pre- 

30 diction coefficient RAM 59 for storing prediction coefficients for HD interpolated pixels, a prediction tap pattern setting 
unit 60 for setting a prediction tap pattern for producing HD interpolated pixels, and a prediction calculation unit 61 for 
producing HD interpolated pixels by executing calculations using the prediction tap pattern and a prediction coefficient. 

A prediction coefficient is read from the prediction coefficient RAM 59 at a location indicated by address data using 
two signals consisting of an ADRC code "C supplied from the one-bit ADRC classification unit 58 and a control signal 

35 CT delayed through the delay circuit 568. The read prediction coefficient is supplied to the prediction calculation unit 
61 . On the other hand, the SD video signal Si sent from the register array 57 is output to the prediction tap pattern set- 
ting unit 60. The prediction tap pattern setting unit 60 sets a prediction tap pattern used for a prediction calculation unit 
61 at the next stage. That is. from the inputted SD signal data S^. the coefficient read from the prediction coefficient 
RAM and the prediction tap pattern executed prediction calculation at the prediction calculation unit 61 at the next stage 

40 are output The prediction calculation unit 61 uses pixel data of the prediction tap pattern arKi the prediction coefficient 
to produce and output HD interpolated pixels by a linear first-order combination. 

Here, four-bit control signal CT generated by the classification unit 5dA, and a class tap pattern selected by the con- 
trol signal CT and used at the classification unit 50B will be described. 

Fig. 1 6 shows a table listing indexes *'I0 to 13" indicating 1 6 (2^) ways of different combinations of the four-bit control 

45 signal CT generated by the classification unit 50A. Rgs. 17A to 17D, 18A to 18D, 19A to 19D, and 20 A to 20D show 
examples of tap pattern configurations of the classification unit 508 at the next stage provided corresponding to the 
indexes "10 to 13". The relation between the control signal CT and the tap pattern, is a relation in which a dass tap pat- 
tern in a large change direction (a direction of large laplacian value) in the spatial direction, and spatial wide of tap 
becomes smaller as a maximum of a laplacian value is large. Further the relation is that a tap pattern is composed of 

50 tap on the same field if a laplacian value in the temporal direction is large, and the tap pattern is composed of tap on a 
different field (for example, a frame) from the tap pattern If the laplacian value in the temporal direction is small. 

Tap patterns shown in Figs. 17A to 17D sire set corresponding to indexes "0000" to "0011". respectively, and 
selected on a large change in the horizontal direction. 

Conrparing the class tap patterns shown in Figs. 17A to 17D. it can be seen that the tap patterns for indexes "0010". 

55 "001 1 ** shown In Figs. 1 7C and 1 7D have pix^s spaced at narrower intervals in the horizontal direction than the tap pat- 
terns for indexes "0000" and "0001** shown in Figs. 17A and 178. whereby the laplacian value of a maximum value in 
the horizontal direction is represented large. That is. the spatial wide of the class tap pattern becomes smaller as a 
laplacian value of a maximum value in the horizontal direction is large, also the spatial wide pf the tap pattern becomes 
larger as the laplacian value of maxinium value in horizontal direction is small. Also, as shown in Figs. 178 and 17D. 
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when bit (fourth bit) in the temporal direction of the indexes "0001 " and "001 1 " is on, the associated class tap pattern is 
positioned in the same field to represent a large change in the temporal direction. On the contrary, as shown in Rgs. 
1 7A and 1 7C. when bit (fourth bit) in the temporal direction of the indexes "0000" and "00 1 0" is off. taps are positioned 
on different fields to represent a small change in the temporal direction. 

5 The class tap patterns for the indexes "OHO" and "0111" shown in Rgs. ISCarxl 1 8D have pixels spaced at nar- 

rower intervals in the vertical direction than the dasstap patterns for the indexes "0100" and "0101" shown in Rgs. ISA 
and 18B, whereby the laplacian value of a maxinium value in the vertical direction is represented large. That is. the spa- 
tial wide of the class tap pattern is smaller as the laplacian value of a maximum value in the vertical direction is large, 
the spatial wide of the dass tap pattern is larger as the laplacian value of a maximum value in the vertical direction is 

10 small. Also, as shown in Rgs. 18B and 18D, when there are changes in the temporal direction of the indexes "0101" and 
"01 11". in other words when the bit (fourth bit) in the temporal direction is on. the class tap pattern is positioned in the 
same field to represent a large change in the temporal direction. On the contrary, as shown in Rgs. 1 8A and 1 8C. when 
the bit (fourth bit) in the temporal direction of the indexes "0100" and "01 1 0" is off. taps are positioned on different fields 
to represent a small change in the temporal direction. 

IS The class tap patterns for the indexes "1010" and "101 1" shown in Rgs. 19C and 19D have pixels spaced at nar- 
rower intervals in the rigfitwardly declining oblique direction than the dass tap patterns for the indexes "1000" and 
"1001" shown in Rgs. 19A and 19B. whereby a laplacian value of a maximum value in the righlwardly declining oblique 
diredion is represented large.. That is. the spatial wide of the dass tap pattern is smaller as the lapladan value of a max- 
imum value in the rightwardly declining oblique direction is large, the spatial wide of the class tap pattern is larger as 

20 the laplacian value of a rreximum value in the rightwardly declining ot>Iique direction is small. Also, as shown in Figs. 
198 and 19D. when bits in the teirporal direction of the indexes "1001 " and "101 1" are on. the associated class tap pat- 
tern is positioned in the same field to represent a large change in the temporal direction. On the contrary, as shown in 
Rgs. 19A and 19C. when bits in the tennporal direction of the indexes "1000" and "1010" are off, taps are positioned on 
different fields to represent a small change in the temporal direction. 

25 The class tap patterns for the indexes "1110" and "1111" shown in Rgs. 20C and 20D have pixels spaced at nar- 
rower intervals in the leftwardly declining oblique direction than the dass tap patterns for the irKlexes "1101" and "1111" 
shown in Rgs. 20A and 208. whereby a laplacian value of a maximum value in the leftwardly declining oblique direction 
is represented large. That is. the spatial wide of the class tap pattern is smaller as the laplacian value of a maximum 
value in the leftwardly declining oblique direction is large, and the spatial wide of the class tap pattern is larger as the 

30 lapladan value of a maximum value in the leftwardly declining oblique direction is small. Also, as shown in Rgs. 208 
and 20D, when bits in the temporal direction of the indexes "1101" and "1 1 1 1 " are on. the associated class tap pattern 
is positioned in the same field to represent a large change in the tennporal direction. On the contrary, as shown in Rgs. 
20A and 200. when bits in the tenpora! direction of the indexes "1 100" and "1110" are off. taps are positioned on dif- 
ferent fields to represent a small change in the temporal direction. 

35 In this way. the dass tap pattern, existing a class tap in large change direction (a direction of the large laplacian 
value) in the spatial direction, is set so that the spatial wide of the laplacian value is smaller as a meiximum value of the 
lapladan value is large. Further, the dass tap pattern is set so that the tap pattern is composed of taps in the same field 
if the lapladan value in the temporal direction is large, and the tap pattern is composed of taps on the different field (for 
example, a frame) from tap pattern if the lapladan value in the temporal direction is small. 

40 Next the foregoing upconvertor 50 of the sixth embodiment and the operation of each unit of the iqpconvertor 50 
will be described. Upon inputting an SD video signal to the first classification unit 50A. the SD video signal is sub- 
jected to Laplacian filtering in a plurality of different level directions in the space by the Laplacian fitters 51 A to 51 D. 
Absolute values aO to a3 of the resulting Laplacian values LO to L3 are calculated by the absolute value circuits 52A to 
52D and outputted therefrom, 

45 Then, the absolute values aO to a3 are supplied to the maximum value detector 53 to detect a maximum value of 
the absolute values of the supplied lapladan values. The direction of an edge in the space of the inputted video signal 
is detected depending upon the direction of the Laplacian value L10 which has detected the maximum value. And the 
value alO indicating the direction of the detected maximum value (detecting direction value of maximum value) js rep- 
resented in two bit form. Further, the maximum value can be compared with a threshold to represent a flatness of the 

50 inputted SD video signal Si. and the value representing the flatness (flatness value) L10 is represented In one bit form. 
Also, a laplacian value L4 in the temporal direction is determined by a lapladan filter 51 E. Abisolute value a4 of the lapla- 
cian value L4 is calculated by an absolute value drcutt 52E and outputted therefrom. The absolute value a4 is com- 
pared with a threshold TH1 by a comparing circuit, thereby output as a value representing a change of the temporal 
diredion (change value of temporal direction) a1 1. In this way, the characteristic of the inputted SD video signal S^ can 

55 be roughly revealed. The classification unit 50A supplies a four-bit control signal OT comprised of two-bit detecting 
diredion value of a maximum value alO. one-bit flatness L10. and one-bit change value of the temporal direction all to 
the second classification unit 508. 

The classification unit 508 at the next stage sets a spatial dass tap pattern on the t>asis of an index "10 to 13" cor- 
responding to the four-t^t control signal OT. More specifically, tiie dassification unit 508 switches the selectors 55A to 
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551 in response to the control signal CT. selects a dass tap pattern determined to the input SD video signal supplied 
through the register array 57, and supplies the selected tap pattern to the ADRC classification unit 58. In this Way, a 
highly accurate classification can be accomplished based on a dass tap pattern reflecting the spatial activity of the SD 
video signal Sv 

5 Also, the SD video signal Si is output from the register array 57 to the a prediction calculation unit 50C at the next 

stage. In a prediction tap pattern unit 60, a prediction coefficients read froma prediction coeffident RAM 59 in accord- 
ance with a control signal CT and an ADRC code "c". and a prediction tap pattern for executing a prediction calculation 
in the prediction calculation unit 61 are set. then the set prediction tap pattern is output to the prediction calculation unit 
61 . In the prediction calculation unit 61 , a prediction calculation is executed by a linear first-order combination using the 

10 prediction coeffidents read from the prediction coeffident RAM 59 in accordance with the control signal CT and the 
ADRC code "c". and the prediction tap pattern supplied from the prediction tap pattern unit 60, thus the HD interpolated 
pixels are output. 

According to the configuration described above, the lapladan filters 51 A to 51 E in the first classification unit 50A 
execute a one-dimensional lapladan operations in plural directions to perform the first classification by synthetically 
15 deciding the values. Next, a class tap pattern of the second dassification unit SOB is set in accordance with the result 
of the first classification unit 50A. Then the second dassification unit 50B performs a dassification using the dass tap 
pattern. Therefore, a highly accurate dassification of the SD video signal S^ can be accomplished reflecting the spatial 
activity of the SD video signal Si. thus producing similar-effects to the foregoing embodiments. 

20 (7) Seventh Emtxxjiment 

Fig. 21, where parts corresponding to those in Fig. 14 are designated the same reference numerals, shows an 
upconv^or 65 according to a seventh emtxxiiment. In similar to the sixth embodiment, in the seventh embodiment, a 
dass tap pattern for a classification of second classif tcation unit 508 in accordance with the first dassification using the 

25 lapladan filters 51 A to 51 E of the first classification 50A is set and a second step classification is executed on the basis 
of the dass tap pattern, thus realizing the dassification at two steps. 

The configuration of the upconvertor 65 in the sixth emt)Odim^ will be described using Fig. 21 . An SD video signal 
S^ inputted from an input terminal IN is supplied to a classification unit 65A. Then the SD video signal S-i supplied to 
the dassification unit 65A is respectively supplied to lapladan filters 51 A to 51 E and a delay unit 56A. Five laplacian 

30 fitters 51 A to 51 E execute laplacian operations on the inputted SD video signal S^ of a block of each frame or each field 
of the supplied SD video signal in the different directions for each filter, and output laplacian values LO to L4. The lapla- 
cian filters 51 A to 51 E use same laplacian filters used in the upconvertor 50 in the sixth emt>odiment shown in Rg. 14. 
The lapladan filter values LO to L4 are respectively supplied to quantizers 66A to 66E at the next stage. 

The quantizers 66A to 66E calculate absdute values of corresponding Laplacians LO to L4, and execute non-linear 

35 quantization to output quantized values qO to q4. For example, if the quantizers 66A to 66E convert the input SD video 
signal S-j. by quantization, into quantized values qO to q4 each of which represents one of two values such as "0" and 
"+1", the inputted SD video signal Si can be classified into a total of 32 (2^ ways of different classes. That is, the quan- 
tizers quantize so as to assign a quantized value "0" in case of small absolute value, and assign a quantized value "1 ** 
in case of big absolute value. Then the quantized values qO to q4 is supplied to a combiner 67, the combiner 67 com- 

40 bines the quarrtized values to generate a five-bit class code, and outputs the f ive-l>it dass code to a second dassifica- 
tion unit 65B as a control signal CT representing first step dassification. 

The dassification unit 65B switches selectors 55A to 551 on the basis of the control signal CT similarly to the sixth 
embodiment, and selects a tap pattern conposed of nine pixels which are SD pixels supplied through a register 57. The 
tap pattern composed nine pixels selected by the selectors 55A to 551 is supplied to an one-bit ADRC dassification unit 

45 58, and the one-bit ADRC dassification unit 58 executes the second step dassification to produce 512 (2^) types of dif- 
ferent dass codes by executing an ADRC operation for the inputted dass tap pattern. In this way. 16384 ways of differ- 
ent classes are acconrplished by a combination of dass of the first-step classification unit 65A and class of the second- 
step dassification unit 65B. The ADRC classification unit 65 generates a nine-bit dass code, and the nine-bit class code 
and the first-step dass code supplied through the delay drcuit 56B are supplied to tiie prediction value calculation unit 

50 65C. The explanation of the calculation unit 85C is omitted because of similarity to the foregoing sixth emtjodiment. 
Also, figure of a class tap pattern used at the dassification unit 65B according to a control signal CT supplied from the 
first dassification unit 65A is omitted. However, as the dass tap pattern, a tap pattern is set so that a spatial wide of a 
tap pattern is smaller and a tap is wider in the direction in which the laplacian value is large as a lapladan value is large. 
Also, the tap pattern is set so that the tap pattern is composed of tap on the same field in case where the lapladan value 

55 in the temporal direction is large, the tap pattern is composed of tap on a different field (for example, frame) in case 
where tiie laplacian value in the terrporal direction is small. By foregoing configuration, it is possft>le to produce similar 
effects to the foregoing embodiments. 
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(8) Eighth Embodiment 

Rg. 22p where parts corresponding to those in Rgs. 4, 5 are designated the same reference numerals, shows a 
classification unit 70 of an upconvertor according to an eighth embodiment. Ah SD video signal Si inputted from an 

5 input terminal IN is parallelly supplied to an ADRC classification unit 71 and a plurality of dass tap selectors 72 (72A to 
72G). The ADRC classification unit 71 executes a classification of the Inputted SD video signal in accordance with 
a pixel level distribution pattern by an ADRC operation using four pixels of around remarked pixel extracted from the 
inputted SD video signal , and supplies a selector 73 with a resulting ADRC code "c". The selector 73 is also supplied 
with tap patterns classified as space classes by the class tap pattern selectors 72A to 72G each for setting a spatial 

10 class tap pattern determined in accordance with the characteristics of the inputted SD video signal Si . 

More specifically, for a level distribution (73A, 73B. 74A. 74B. 75A. 75B. 76A, 768. 77A. 778. 78A. 788. 79A. 798. 
80A, 808) Indicated by each ADRC code "c" resulting from one-bit ADRC operation performed on every four pixels for 
around the remarked pixel of the inputted SD video signal S-j, each of the selectors 72 A to 72G sets a class tap pattern 
(81. 82. 83, 84. 85, 86. 87) for representing the signal characteristics corresponding to the associated level distribution, 

15 as shown in Rgs. 23A, 238, 23C, 23D. 24A. 248. and 24C. 

For example, level distributions 73A and 738 indicated by ADRC codes "c" respectively exhibit level distribution 
characteristics in a rightwardly declining direction (the direction of a diagonal from the upper left end to the lower right 
end on the plane of the drawing). Since an edge of the image Is thought to exist in that direction, a class tap pattern 81 
In the rightwardly declining oblique direction is corresponded to the level distributions 73A and 738. SImilariy, since level 

20 distributions 74A and 748 Indicated by ADRC codes "c" respectively exhibit level distribution characteristics in a left- 
wardly declining direction (the direction of a diagonal from the upper right end to the lower left end on the plane of the 
drawing), a class tap pattern 82 In the leftwardly declining oblique direction is corresponded to the level distributions 
74A and 748. In addition, since level distrtoutions 75A and 758, 76A and 768 indicated by ADRC codes "c" exhibit level 
distribution characteristics offset on the left side and on the right side, respectively, class tap patterns 83 and 84 having 

25 prediction taps offset on the left side and on the right side are corresponded to the level distributions 75A and 758. 76A 
and 768. respectively. 

Further, since level distributions 77A and 778. 78A and 788 indicated by ADRC codes "c" exhibit level distribution 
characteristics offset on the upper side and on the lower side, respectively, class tap patterns 85. 86 having class taps 
offset on the upper side and on the lower side are corresportded to the level distritxitions 77A and 778, 78A and 788. 
30 respectively. Since level distributions 79A and 798. 80A and 808 indicated by ADRC codes "c" exhibit regular level dis- 
tribution characteristics, a dass tap pattern 87 using all class tap is corresponded to the level distributions 79A and 798. 
80A and 808. respectively. 

In this way, a class tap pattern plO is selected In the selector 73 in accordance with the ADRC codes "c" from the 
class tap patterns set in tiie tap pattern selectors 72 A to 72G. and supplied to an ADRC classification unit 26 at the next 

35 stage. The ADRC classification unit 26 uses the selected prediction tap pattern plO to perform a one-bit ADRC opera- 
tion on the SD video signal Si and supplies a prediction coefficient ROM 14 with the resulting ADRC code dO as 
address data for reading corresponding prediction coefficients. 

Next, the operation of the classification unit 70 In the upconvertor In eighth embodiment will be described. 

The inputted video signal Si Is supplied to first ADRC clasaf ication unit 71 , which executes an one-bit ADRC oper- 

40 ation using four pixels for around the remarked pixel and produces a dass code "c**. Also, the inputted SD video signal 
S-i Is classified into a plurality of dass tap patterns 81 to 87 reflecting the signal characteristics in the class tap pattern 
selectors 72A to 72G. Then a dass tap pattern plO is selected from the plurality of dass tap patterns 81 to 87 in accord- 
ance with the class code "c" supplied from the first ADRC classification unit 71 , and output to an ADRC dassification 
unit 26 at the next stage. The ADRC dassification unit 26 executes one-bit ADRC operation on the inputted SD video 

45 signal S-i formied the selected dass tap pattern pi 0 to generate a dass code dO and supplies the class code dO to the 
prediction coefficient ROM 14 as address data. 

According to the configuration described above, for produdng address data in order to seled prediction coeffi- 
cients, a class tap pattern is selected in accordance with the spatial activity of an inputted SD video signal S^ detected 
by the ADRC classification unit 71 at tiie first stage. Then the second-step ADRC dassification Is executed by the 

50 ADRC classification unit 26 using the selected dass tap pattern plO. so tiiat the class code dO reflecting tiie spatial 
activity of the Inputted SD video signal S-i can be generated, thus produdng similar effects to the foregoing emtx)di- 
ments. 

In addition, according to the embodiment described atx>ve, a prediction tap pattern used for a prediction calculation 
Is changed in accordance with the spatial activity of the inputted SD video signal S^. so that the calculation processing 
55 can be reduced when there are many prediction taps. 

Further, as the first errlbodiment number of quantization bits in second-stage ADRC dassification unit 26 can be 
switched on the-t)asis of the outputted signal outputted from first-stage ADRC classification unit 71 . 
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(9) Ninth Embodiment 

Fig. 25 shows an upconvertor 90 according to a ninth embodiment. The upconvertor 90 first execute a rough clas- 
sification for an intra-space activity in accordance with a one-bit ADRC operation in a first-stage classification unit and 
5 then executes a multi-bit ADRC operation for a detailed classification in a second-stage classification unit. 

In the upconvertor 90. a SD video signal Si inputted through an input terminal IN is respectively output to a first- 
stage block 91. delay circuits 97 and 101. The first-stage block 91 extracts a block of n x m pixels (for example. 5x5 
pixels in Fig. 26) centered on a remarked pixel (indicated by "® " in Rg. 26) in a current frame or field of an SD video 
signal in a first-stage block 91 as shown in Fig. 26, and supplies a one-bit ADRC classification unit 92 with resulting 
10 block data b1. 

The one-bit ADRC classification unit 92 executes a one-bit ADRC operation on the block data b1 consisting of 5 x 
5 pixels and supplies a resulting ADRC code clO to a ROM 94. after passing it through a delay circuit 93A for adjusting 
the fining. The one-bit ADRC classification unit. 92 further supplies a comparator circuit 95 with a dynamic range DR 
calculated when the ADRC code clO was derived. Simultaneously with this, the one-bit ADRC classification unit 92 

15 supplies a multi-bit ADRC classification unit 96 with the dynamic range DR and a minimum pixel level MIN. 

The comparator circuit 95 compares the dynamic range DR with a threshold TH and supplies the conrparison result 
CR to the ROM 94 through a delay circuit 93B. The multi-bit ADRC classification unit 96 does not execute a classifica- 
tion based on the comparison result CR if the dynamic range DR is smaller than the threshold TH (CR equals "0"). On 
the other hand, the multi-bit ADRC classification unit 96 executes the classification if the dynamic range DR is larger 

20 than the threshold TH (CR equals "1"). However, in case of this embodiment, the multi-bit ADRC classification unit 96 
is generally executed. Therefore in a ROM 94 at subsequent stage, an ADRC code c1 1 from tiie multi-bit ADRC clas- 
sification unit 96 is ignored, thereby it is considered not to execute tiie multi-bit ADRC classification unit 94. Also, so as 
not to execute the multi-bit ADRC classification, the comparison result CR is supplied to the multi-bit ADRC classifica- 
tion unit 96 as shown a broken line in Rg. 25. thereby it has to be controlled not to make it execute the multi-bit ADRC 

25 classification unit 96. 

The SD video signal S-i delayed through the delay circuit 97 to have its timing adjusted is supplied to a second- 
stage block 98 which defines, for example, a block data h2 of nine pixel data consisting of 3 x 3 pixels including a 
remarked pixel, as shown in Rg. 26, which is supplied to the multi-bit ADRC classification unit 96. The mufti-bit ADRC 
classification unit 96 uses the dynamic range DR and the minimum pixel level MIN calculated in one-bit ADRC ctassifi- 

30 cation unit 92 to classify the block data b2. and supplies a resufting ADRC code c1 1 to the ROM 94. 

As shown in Rg. 27. a table has been previously produced by learning for dass code dO having a hierarchical struc- 
ture in accordance with the classification of ADRC code clO derived as the resuH of the one-bft ADRC classification unit 
92 and ADRC codes c1 1 derived as the result of the multi-bit ADRC classification unit 96. and has b>een stored in the 
ROM 94. The classification unit 90 reads a class code dO in accordance witii ADRC codes clO and c1 1 derived by the 

35 one-bit ADRC classification unit 92 and the mufti-bit ADRC classification unrt 96 based on the connparison resuft CR, 
and supplies the read dass code dO to a prediction coefficient RAM 99 at the next stage. Likewise, prediction coeffi- 
cients derived by learning are stored in the prediction coefficient RAM 99 in accordance wfth the class codes dO 
arranged in a hierarchical structure. 

A set of prediction coefficients are sequentially read from the prediction coeffident RAM 99 with the dass code dO 

40 used as address data, and supplied to a prediction calculation unft 100 for calculating HD interpolated pixels by execut- 
ing a product sum calculation witii the SD video signal S^ The prediction tap setting unft 102 is supplied with the SD 
video signal S^ delayed through a delay circuft 101 . and outputs a prediction tap pattern, as shown in Rg. 28, used in 
the prediction calculation unrt 100 to the prediction calculation unit 100. The prediction tap setting unft 102 produces 
interpolated pixels using a prediction tap pattern as shown in Rg. 28. In this way. the prediction calculation unit 100 exe- 

45 cutes a linear prediction calculation witii the group of prediction coeffidents corresponding to the SD video signal S-i to 
produce interpolated pixels. 

Next, the upconvertor 90 according to the ninth emlxxJiment and the operation of each unrt of the upconvertor 90 
will be described. 

an SD video signal Si inputted to the first-stage block 91 of the upconvertor 90 is extracted in the unit of 5 x 5 pixel 
50 blocks in which a remarked pixel is the center, and each pixel block is classftied in tiie one-bft ADRC classification unft 
92. Then, a dynamic range DR of each pixel block outputted from the one-bit ADRC dassification unrt 92 is compared 
witii a predetermined tiireshold TH in the connparator drcurt 95. At this time, ft the dynamic range DR is larger than tiie 
threshdd TH. tiiemulti-bft ADRC dassification unit 96 is executed. Then, a class cod dO is read from tiie ROM 94 based 
on the results of the one-bft ADRC classification unrt 92 and the muHi-brt ADRC classification unrt 96. On the contrary. 
55 ft the dynamic range DR is smaller than the threshold TH, a class code Is read from the ROM 94 fc>ased only on the 
resuft of the one-bft ADRC classification unft 92 witiiout executing the mufti-brt ADRC dassftkiation unft 96. 

In tills way, tiie class code dO is read from the ROM 94 in accordance witii the results of the one-l^t ADRC dassi- 
fication unrt 92 and/or the multi-bit ADRC dassification unit 96. and supplied to the prediction coefficient RAM 99 as 
address data. The intra-space activrty is roughly evaluated in the first-step classification and a detailed classification is 
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executed in accordance with the intra-space activity at the second step, so that the inputted SD video signal can be 
appropriately classified reflecting the intra-spatial activity thereof. 

A set of prediction coefficients are read from the prediction coefficient RAM 99 in accordance with the class code 
do and supplied to the prediction calculation unit 100. The prediction calculation unit 100 performs a prediction calcu- 
5 lation using the prediction coefficients on the prediction tap pattern selected in the prediction tap pattern selector 102 
to produce and output HD interpolated pixels. 

According to the conf iguratton described attove, a rough classificatidn is executed at the first step and a detailed 
classification is executed at the second step, so that an input SD video signal can be more appropriately classified 
In detail at the second step in accordance with the signal characteristics of the input SD video signal S^ revealed by the 
10 flrst-st^ classification. This produces similar effects to the foregoing emtxxliments. In addition, if the classification is 
completed only at the first step, the entire classification processing can be reduced. 

(10) Other Errt)Odiments 

15 While the foregoing embodiments have dealt with the case where an ADRC classification technique is used as a 
classification method based on corrpressed data of input video signal. However, the present invention is not limited to 
this, but classification can be executed by conpressing data using other techniques Including, for exarrple, those 
employing DPCM (Differential Pulse Code Modulation), VQ (Vector Quantization) and MSB (Most Significant Bit), bina- 
rization, discrete cosine transform (DOT), and so on. 

20 Also, while the foregoing enrAxxJiments have dealt with the case where the classification unit 12 executes the clas- 
sification at two steps or at three steps. However, the present invention is not limited to tiie classification completed by 
such particular numbers of steps, but the classification can be executed at a larger nuvrber of steps If the adaptability 
is successively increased at subsequent steps after a rough classification at the first step. With a larger number of 
steps, a more accurate classification can be accomplished. 

25 Further, while the foregoing embodiments have dealt with the case where a two<limensional non-separable fitter is 
used as an upconvertor. However, the present invention Is not limited to this type of upconvertor, and can be applied to 
an upconvertor 1 10 comprising a vertical/horizontal separable filter configuration as illustrated in Fig. 29, where parts 
corresporxling to those in Rg. 4 are designated the same reference numerals. 

In the upconvertor 110. an SD video signal 8^ inputted through an input terminal IN is first supplied to a dassifica- 

30 tion unit 1 2 and a prediction calculation unit 111. The prediction calculation unit 1 1 1 is divided into two sets: a vertical 
prediction calculation unit 11 1 A and a horizontal prediction calculation unit 11 1B conresponding to positions mode 1 
and mode 2 on a scanning line, respectively, and a vertical prediction calculation unit 1 1 1 C and a horizontal prediction 
calculation unit 1 1 1 D corresponding to positions mode 3 arKi mode 4 on a scanning line, respectively. The classification 
unit 12. as adapting tiie dassification of the foregoing embodiment, generates a dass code dO in accordance with the 

35 inputted SD video signal S-). which is supplied to a prediction coeffiderrt ROM 112 serving as a storage means which 
previously stores tap prediction coeffidents. The prediction coefficient ROM 112 is divided Into a vertical coefficient 
ROM 1 1 2A for storing vertical components of tap prediction coeffidents and a horizontal coeff ident ROM 1 1 2B for stor- 
ing horizontal components of the tap prediction coeffidents. The class code dO is supplied to each of the vertical coef- 
ficient ROM 1 12A and the horizontal coefficient ROM 1 12B. 

40 First a vertical prediction coefficient d6 outputled from the vertical coefficient ROM 1 1 2A is supplied to vertical pre- 
diction calculation units 1 1 1 A arKi 1 1 1 C. The vertical prediction calculation units 1 1 1 A arKi 1110 generate vertical esti- 
mated values d7 and d8 by a product sum calculation of the inputted SD video signal Sf and the vertical prediction 
coeffiderrt d6. The vertical estimated values d7 and d8 are supplied to horizorrtal prediction calculation units 1 1 1 B and 
1 1 1 D at the next stage, respectively. 

45 A horizontal prediction coefficient d9 generated from the horizontal coefficient ROM 1 12B is supplied to horizontal 
prediction calculation units 11 IB and 111D. The horizontal prediction calculation units 11 IB and 11 ID produce HD 
pixel signals dIO and d1 1 by performing a product sum calculation of the horizontal prediction coefficient d9 and the 
vertical estimated values d7 and d8. TTie HD pixel signals dIO and d1 1 are selectively supplied to a selector 15 where 
it is appropriately rearranged, whereby an HD signal 82* which Is a final output, is outputted from an output terminal 

50 OUT 

Also, while the foregoing embodiments have dealt with the case where prediction coefficients, each representing a 
correlation between a remarked SD pixel and transmitted pixels around the remarked pixel, are used to produce HD pix- 
els around the remarked pixel from SD pixels. However, the present invention is not limited to this form of producing HD 
pixels, but predicted values for HD interpolated pixels can be previously set for each class instead of the prediction 
55 coefficients, and stored in a storage means. Conversion of an SD video signal Into an HD video signal using predicted 
values may be performed by an upconvertor 1 15 as shown in Rg. 30, where parts corresponding to those in Rg. 4 are 
designated the same reference numerals. 

In the upconvertor 1 15. an SD video signal S^ is supplied to a dassification unit 12 through an Input terminal IN. 
The classification unit 12 generates a class code dO based on the characteristics of SD pixels around HD interpolated 
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pixels which are interpolated pixels to be newly produced, and supplies the class code dO to predicted value ROMs 
1 1 6A to 11 6D. The predicted value ROMs 1 1 6A to 1 1 6D store predicted values constituting prediction data for HD inter- 
polated pixels previously calculated by learning in correspondence to the class code dO for each class. Predicted values 
d20 to d23 for HD interpolated pixels are read from the predicted value ROM 116 with the class code dO used as 
address data, and outputted through a selector 15 to an output terminal OUT In this way, it is possible to produce a high 
resolution video signal having the predicted values 620 to d23 used as HD interpolated pixels inserted into signal pixels 
constituting the inputted video signal . 

A first method for calculating the predicted values may be a learning method using a weighted average technique. 
The weighted average technique classifies remarked pixels using SD pixels around the remarked pixels, and divides a 
pixel value of renrrarked pixels (I.e.. HD pixels) added up for each class by a frequency incremented in accordance with 
the number of remarked pixels. These operations are performed on a variety of images to derive predicted values. 

A second method for calculating predicted values may be a learning method by normalization. This learning 
method first forms a block comprising a plurality of pixels including a remarked pixel, and utilizes a dynamic range in the 
block to normalize a value calculated by subtracting a reference value of the block from the pixel value of the remarked 
pixel. Next, an accumulated value of the normalized values is divided by an accunujiated frequency to derive a pre- 
dicted value. 

Further, in the foregoing embodiment, an irrtra-activity is evaluated for an inputted SD video signal, and tap pattern 
for executing dassificatibhs is selected on the basis of a result of the evaluation. However, the preserrt invention is not 
limited thereto and an intra-activity is evaluated for an inputted SD video signal, and a prediction tap pattem calculated 
by a linear first-order comtMnation of a prediction coefficient in a prediction calculation unit on the basis of the result of 
the evaluation. In this case, for example, a control signal CT supplied from the classification unit 50A shown in Fig. 14 
is supplied to the prediction tap pattern unit 60 in the calculation unit 50C. 

Further, in the foregoing embodiment a prediction coefficient of a prediction coefficient memory is read on the 
basis of tx>th a class code supplied from first-step classification unit and a dass code supplied from second-step clas- 
sification unit However, the present invention is not limited thereto and a prediction coefficient of a prediction coefficient 
memory can be read uiing either a class code supplied from first-step classification unit or a dass code supplied from 
second-step classification unit 

Furthermore, in the foregoing embodiment, an SD video signal is converted into an HD video signal. However, the 
present invention is not limited thereto and the SD video signal can be applied to the generation of interpolated pixels 
for enlarging and changing an image. Also, the SD video signal can be applied to a signal converting apparatus, such 
as an convertor for converting signals of NTSC method into signals of PAL (phase alternation by line), .for converting 
signals of few number of pixels into signals of many number of pixels. Also, the SD video signal can be applied to YC 
s^arating apparatus for generating higher accuracy signals than former signals. 

According to embodiments of the present invention as described atove, an intra-space activity is evaluated and 
classified for each block of an inputted video signal, and each block of the input video signal is dassif led at an appro- 
priate number of steps in accordance with an activity code generated as a result of the dassif ication. Thus, the activity 
code generated at the first step and the result of dassif ications at previous steps are reflected to dassifications at next 
and subsequent steps, so that a highly accurate dassification can be accomplished for the input video signal, and 
highly accurate interpolated pixels can be predicted and produced for a low resolution input image, using prediction 
coefficients or predicted values based on the results of the classifications, thereby enabling to realize the signal corrver- 
tor and signal converting method which can produce a high resolution video signal. 

In the limits of not deviating an opinion of the present invention, various changing and application example are con- 
sidered. Therefore, the present invention is not limited by or to the described embodiments. 

While there has been described in connection with the prefen-ed embodiments of the invention, it will be obvious to 
those skilled in the art that various changes and modifications may be aimed, therefore, to cover in tiie appended clainrts 
all such changes and nrodifications as fall witiiin the true scope of tiie invention. 

Claims 

1 . A signal converting apparatus for converting an inputted first video signal into a second video signal different from 
the first video signal, comprising: 

means for evaluating an intra-space activity for said first video signal to generate an activity code; 

means for executing stepwise dassifications based on said activity code to generate a dass code based on 

the result of said dassification; 

a prediction coefficient memory stored with prediction coeffidents for predictively producing said second video 
signal using said f irst video signal; and 

means for predictively calculating said first inputted video signal using said prediction coeffident read from sakJ 
prediction coeffiderrt memory according to said activity code and/or said class code to produce said second 
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video signal. 

2. The signal converting apparatus according to claim 1 , wherein said first video signal is a low resolution video signal, 
and said second video signal is a high resolution video signal which is higher resolution than said low resolution 

5 video signal. 

3. The signal converting apparatus according to claim 1 . wherein said sebond video signal is a video signal which has 
the nunrdser of pixels more than said first video signal. 

10 4. The signal converting apparatus according to claim 1 , wherein said activity code producing means evaluates an 
intra-space activity and a tenporal direction activity for said first video signal to generate an activity code. 

5. The signal converting apparatus according to claim 1 , wherein said dass code producing means sets a plurality of 
different pixel patterns for said first video signal to select a pixel pattern from the set plurality of pixel patterns in 

75 accordance with said activity code, and classifies said first video signal using the selected pixel pattern in order to 
generate a class code. 

6. The signal converting apparatus according to daim 1 , wherein said activity code producing means evaluates an 
intra-space activity using a dynamic range of each pixel in a neight>oring region of a remarked pixel in said first 

20 video signal in order to generate an activity code. 

7. The signal converting apparatus according to daim 1, wherein said activity code producing means evaluates an 
intra-space activity in accordance with a level distribution of quantized value obtained based on a dynamic range 
defined by pixels in a neighboring region of a remarked pixel in said first video signal in order to generate an activity 

25 code. 

8. The signal converting apparatus according to daim 1. wherein said activity code producing means evaluates an 
intra-space activity using a standard deviation obtained from signal distribution of each pixel in a neighboring region 
of a remarked pixel in said first video signal in order to generate an activity code. 

30 

9. The signal converting apparatus according to daim 1, wherein said activity code producir^ means evaluates an 
intra-space activity in accordance with frequency distribution of quantized value obtained based on a dynamic 
range defined by pixels in a neighboring region of a remarked pixel in said first video signal in order to generate an 
activity code. 

35 

10. The signal converting apparatus according to daim 1, wherein said activity code produdng means evaluates an 
Intra-space activity in accordance with frequency distribtution of differences of respective adjacent pixel values for 
each pixel in a neighboring region of a remarked pixel in said first video signal in order to generate an activity code. 

40 11. The signal converting apparatus according to daim 1, wherein said activity code producing means evaluates an 
intra-space activity based on laplacian values obtained in respective intra-space different direction using laplacian 
fitters in order to generate an activity code. 

12. The signal converting apparatus according to daim 1, wherein said activity code producing means evaluates an 
45 Intra-space activity and a temporal direction activity based on lapladan values obtained intra-space and temporal 

direction different directions using laplacian filters in order to generate an activity code. 

13. The signal converting apparatus according to daim 1, wherein said activity code producing means evaluates an 
intra-space activity in accordance with level distribution of each pixel in a neightx>ring region of a remarked pixel in 

50 said first video signal In order to generate an activity code. 

14. The signal converting apparatus according to daim 1 . wherein said class code producing means classifies said first 
video signal by quantizing said first video signal on the basis of an dynamic range defined by each pixel In a neigh- 
boring region of a remarked pixel in said video signal in order to generate a dass code. 

55 

15. The signal converting apparatus according to daim 14, wherein said class code produdng means adaptively 
changes a level resolution capability in case of quantizing said first video signal in accordance with said activity 
code. 
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16. The signal converting apparatus according to daim 4. wherein said class code producing means sets a pixel pat- 
tern for a wide region, a pixel pattern for a narrow region as compared with said wide region, and a standard pixel 
pattern corre^ond to between said wide region and said narrow region, for said first video signal. 

5 17- The signal converting apparatus according to claim 1 . wherein: 

said activity code producing means quantizes each pixel in a neighboring region of a remarked pixel in said 
first video signal to compress data and evaluates an intra-space activity in accordance with characteristics of a level 
distribution for the quantized value in order to produce said activity code; and 

said class code producing means sets an adaptive pixel pattern in accordance with said activity code, and 
10 sets predetermined number of bits for each pixel of the pixel pattern in accordance with said activity code to quan- 
tize it 

18. A signal converting apparatus for converting an inputted first video signal into a second video signal different from 
the first video signal, comprising: 

15 

means for evaluating an intra-space activity for said first video signal to generate an activity code; 

means for performing stepwise classification on the basis of said activity code to generate a class code on the 

basis of the result of the dassiftcation; and 

means, having a prediction value storing memory stored with a prediction value generated as an interpolated 
20 pixel signal for said first video signal, for reading and outputting a prediction value corresponding to said activity 

code and/or said dass code. 

19. The signal converting apparatus according to claim 18. wherein said first video signal Is a low resolution video sig- 
nal, and said second video signal is a high resolution video signal which is higher resolution than said low resolution 

25 video signal. 

20. The signal converting apparatus according to claim 18. wherein said second video signal is a video signal which 
has the number of pixels more than said first video signal. 

30 21. The video signal converting apparatus according to claim 18, wherein said activity code producing means evalu- 
ates an intra-space activity and temporal direction activity for said first video signal in order to generate an activity 
code. 

22. The video signal converting apparatus according to claim 18. wherein said class code produdng means sets a plu- 
35 rality of different pixel patterns for said first video signal to select a pixel pattern from the set plurality of pixel pat- 
terns in accordance with said activity code, and classifies said first video signal using the selected pixel pattern in 
order to generate a dass code. 

23. A video signal corTverting method for converting an inputted first video signal into second video signal different from 
40 first video signal, comprising the steps of: 

evaluating an intra-space activity for said first video signal to generate an activity code; 

performing stepwise dassification teased on said activity code to generate a class code based on the result of 

the dassification; 

45 reading a prediction coeff ident stored in a prediction coefficient memory for predictively producing said second 

video signal, using said first video signal in accordance with said activity code and/or said class code; 
performing a prediction calculation for said first inputted video signal using the read prediction coefficient; and 
outputting a prediction calculation value as said second video sgnal. 

50 24. The signal converting method according to daim 23. wherein said first video signal is a low resolution video signal, 
and said second video signal is a high resolution video signal which is higher resolution than said low resolution 
video signal. 

25. The signal converting method according to daim 23. wherein said second video signal is video signal which has 
55 the number of pixels more than said first video signal. 

26. The signal converting method acooiding to claim 23, wherein said activity code producing step evaluates an intra- 
space activity and a temporal direction activity for said first video signal. 
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27. The signal converting method according to claim 23, wherein said class code producing step sets a plurality of dif- 
ferent pixel patterns for said first video signal to select a pixel pattern from the set plurality of pixel patterns in 
accordance with said activity code and classifies said first video signal using the selected pixel pattern. 

5 28. The signal converting method according to claim 23. wherein said activity code producing step evaluates an intra- 
space activity using a dynamic range for each pixel in a neighboring region of a remarked pixel in said first video 
signal. 

29. The signal converting method according to claim 23, wherein said activity code producing step evaluates an intra- 
10 space activity in accordance with a level distribution for quantized value obtained l>ased on a dynamic range 

defined by pixels in a neighboring region of a remarked pixel in said first video signal. 

30. The signal converting method according to claim 23, wherein said activity code producing step evaluates an intra- 
space activity using a standard deviation obtained from a signal distribution of each pixel in a neighboring region of 

75 a remarked pixel in said first video signal. 

31. The signal converting method according to claim 23, wherein said activity code producing step evaluates an intra- 
space activity in accordance with a frequency distribution for a quantized value obtained based on a dynamic range 
defined by pixels in a neightx)ring region of a remarked pixel in said first video signal. 

20 

32. The signal converting method according to claim 23, wherein said activity code producing step evaluates an intra- 
space activity in accordance with a frequency distribution of differences of re^ective adjacent pixel values for each 
pixel in a neighboring region of a remarked pixel in said first video signal. 

25 33. The signal converting method according to claim 23, wherein said activity code producing step evaluates an intra- 
space activity based on a lapladan value obtained in respective intra-space different directions using laplacian fil- 
ters. 

34. The signal converting method according to claim 26, wherein said activity code producing step evaluates an intra- 
30 space activity and a temporal direction activity based on a laplacian value respectively obtained in intra-space 

direction and t&nporal direction different directions using laplacian filters. 

35. The signal converting method according to claim 23. wherein said activity code producing step evaluates an intra- 
space activity in accordance with a level distribution in a neight>oring region of a remarked pixel in said first video 

35 signal. 

The signal converting method according to claim 23. wherein said class code producing step classifies said first 
video signal by quantizing said first video signal based on a dynamic range defined by each pixel in a neightx^ring 
region of a remarked pixel in said first video signal. 

The signal converting method according to daim 36. wherein said dass code produdng step changes a level res- 
olution capability in case of quantizing said first video signal in accordance with said activity code. 

33. The signal converting method according to claim 26. wherein said dass code produdng step sets a pixel pattern 
45 for a wide region, a pixel pattern for a narrow region as compared with said wide region, and a standard pixel pat- 
tern conresponding to between said wide region and said narrow region, for said first video signal. 

The signal converting method according to claim 26. wherein: 

said activity code produdng step quantizes each pixel in a neighboring region of a remarked pixel in sakJ first 
video signal to compress data and evaluates an intra-space activity in accordance with characteristics of a level das- 
tribution for the quantized value; and 

said class code producing st^ sets an adaptive pixel p>attern In accordance with said activity code, and sets 
predetermined number of bits for each pixel of the pixel pattern in accordance with said activity code to quantize it. 

55 40. A signal converting method for converting an inputted first video signal into a second video signal different from the 
first video signal, corrprising the steps of: 

evaluating an intra-space activity for said first video signal and outputting an activity code; 

performing stepwise classifications based on said activity code and outputting a class code based on a result 
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of the classification: 

reading a prediction value stored in a prediction value memory in accordance with said activity code and/or 
said class code; and „ . 

outputting said prediction value produced as an interpolated pixel signal for said first video signal. 

5 

41. The signal converting apparatus according to claim 40, wherein said first video signal is a low resolution video sig- 
nal, and said second video signal is a high resolution video signal which is higher resolution than said low resolution 
video signal. 

10 42. The signal converting method according to claim 40, wherein said second video signal is a video signal which has 
the number of pixels more than said first video signal. 

43- The signal converting method according to claim 40, wherein said activity code producing step evaluates an intra- 
space activity and a temporal direction activity for said first video signal. 
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44. The signal converting method according to claim 40. wherein said activity code producing step sets a plurality of 
different pixel patterns for said first video signal, selects a pixel pattern from the set plurality of pixel patterns in 
accordance with said activity code, and classifies said first video signal using the selected pixel pattem. 
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(54) Signal converting apparatus and signal converting method 

(57) The signal converting apparatus and method 
predictively produces highly accurate interpolated pix- 
els in accordance with a classification which precisely 
reflects a variety of signal characteristics of inputted 
vkJeo signals to provide a high resolution video signal. 
An activity is evaluated and classified (12) for each 
biock of an inputted video signal (81). Stepwise classifi- 
cations (p0-p2), (Rg 5. 22-24) are executed on each 
block of the inputted video signal (81) and a classifica- 
tion (CI) is selected in accordance with an activity code 
(cO) obtained as a result of the activity classification 
(21). In this way, the accuracy of subsequent classifica- 
tions can be increased, reflecting the activity character- 
istic of each block of the inputted video signal (81), thus 
achieving, as a whole, a highly accurate classification of 
the inputted video signal (81). Appropriate prediction 
coefficients (d1) stored in a ROM (14) are read out 
teased on the activity code (cO) and the class code (c1) 
for each block of the inputted video signal (SI). The 
coefficients are used in calculators (13) to produce 
highly accurate interpolated pixel values (d2-d5) which 
are selected by a selector 15 to provide a high resolu- 
tion video signal (S2). 
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